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May 23, 2019 

The Honorable Connie M. Leyva 
Chair, Senate Education Committee 
State Capitol, Room 2083 
Sacramento, CA 95814 

Dear Senator Leyva: 

I am writing to share my serious concerns regarding Assembly Bill 751, the "Pathways to College 
Act." For over 35 years, I served on the faculty of the Stanford University Graduate School of 
Education. I am recently retired, now the Jacks Family Professor of Education, Emeritus. I have a 
Ph.D. from the University of Chicago, Department of Education, with a concentration in 
measurement, evaluation, and statistical analysis. I am a recognized expert in educational 
research, and in particular in educational testing and measurement, as shown on my 
Curriculum Vitae (available at https://people.stanford.edu/haertel/curriculum-vitae) . 

My history with testing issues in the State of California reaches back to membership on 
Technical Advisory Committee for the California Assessment Program in the 1980s, and I served 
on the Advisory Committee for the Public Schools Accountability Act of 1999 from the time that 
committee was first formed until it ceased to function in 2015. In 2007,1 received the Lifetime 
Achievement Award from the California Educational Research Association. I currently serve on 
the Technical Design Group working with the CDE's Analysis, Measurement, and Accountability 
Reporting Division, and I also serve on the Technical Advisory Committee for the Smarter 
Balanced Assessment Consortium. In this letter, I am expressing my own views. I am not 
representing the views of Stanford University. 

I am deeply concerned about the implications of AB 751 for California's school accountability 
system and ultimately, for the quality of education offered to public-school students in our 
great state of California. There is a natural desire to reduce testing burden, and on the face of 
it, using the same test for multiple purposes seems like a sensible solution. Since many high 
school students are already taking the ACT or the SAT, for example, why should they also have 
to take the llth-grade Smarter Balanced assessments? Moreover, encouraging more students 
to take college entrance exams seems like a good thing—As suggested by the name of the 
"Pathways to College Act," mandating one of these tests might encourage college application 
and college enrollment. In addition, high school students might be more motivated to try hard 
on a college entrance exam than on Smarter Balanced assessments for which they perceived 
fewer personal consequences. 

As a psychometrician who has devoted my entire professional career to understanding and 
improving achievement tests and accountability testing programs, I certainly share the 
concerns of policy makers and the public over testing burden, access to higher education, and 
student motivation, but despite these concerns, I believe that AB 751 is very unwise. The 
authors of the bill were clearly thoughtful and well-intentioned, having included numerous 
sensible requirements concerning rigor, coverage, inclusiveness, score reporting, and other 
matters, but as a practical matter, these paper requirements would be quite easy to satisfy. In 

485 Lasuen Mall, Stanford, CA 94305-3009 T 650.725.1251 F 650.725.7412 http://people.stanford.edu/haertel/ 



Haertel letter re AB 751 - Page 2 


the end, what the bill comes down to is a proposal to take two (or more) very different 
examinations, developed according to different specifications to serve different purposes, 
administered under different conditions (including different motivational contexts), and then to 
treat the resulting scores as interchangeable. We may wish such a thing were possible, but it 
simply cannot be done. Methods for expressing scores from different tests on a common scale, 
or for using and interpreting them as if interchangeable, fall far short of providing the degree of 
comparability California's state-of-the-art school accountability system would require. 

I cannot help but think back to the short-lived "Pupil Testing Incentive Program" (PTIP) enacted 
in California over 20 years ago, after the demise of the California Learning Assessment System 
(CLAS) and before the enactment of the Standardized Testing and Reporting (STAR) program. 
PTIP was supposed to, in fact was mandated to, produce comparable scores across a menu of 
approved off-the-shelf achievement tests that districts could choose from. I was among the 
testing experts who were concerned at that time that the legislation's aspirations had gotten 
out ahead of what the testing field would be able to deliver, and that in fact proved to be the 
case. The effort was abandoned within two years. I am afraid that despite its good intentions, 
AB 751 likewise reaches well beyond the range of technical feasibility. 

The comparability requirements included in AB 751 might sound rigorous, but in practice, it 
would be very easy to satisfy "alignment," depth-and-breadth, and similar technical criteria 
despite material differences in item formats, content coverage and weighting, and ultimately in 
the constructs assessed by the Smarter Balanced tests versus some college entrance exam. (In 
the PTIP program, a seemingly rigorous test review process was in practice easily 
circumvented.) Scores on different tests have different meanings, even if the names of the tests 
are the same or similar, even if the tests are nominally aligned to the same content 
frameworks. Even if it were possible to scale the scores from such different tests so as to 
provide adequate comparability at the level of individual students (which is far from 
guaranteed), when scores were aggregated to the level of schools, or demographic subgroups 
within schools, small biases and differences in score meaning, as well as differences in the tests' 
precision and error structures, would take on much greater importance, resulting in deeply 
flawed comparisons across schools participating in different testing regimes. 

In addition, many colleges and universities in California have already agreed to accept specified 
performance levels on the Smarter Balanced tests as evidence that students can bypass 
remedial courses or waive co-requisites. As Smarter Balanced scores are increasingly accepted 
as evidence of readiness by California's community colleges and CSU institutions, llth-grade 
Smarter Balanced scores will become even more valuable and meaningful for our high school 
students. AB 751 would deprive students in opt-out districts not only of valuable information 
concerning their own college readiness, but also of tangible evidence they could otherwise use 
to jumpstart their college careers. 

Finally, I am seriously concerned with "wash-back" or "up-stream" effects of abandoning the 
Smarter Balanced assessments for high school students. While we can never rely solely on 
incentives to "teach to the test" as a tool for curriculum reform, we can at least choose tests 
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with the virtue that sound test preparation is also sound instruction. In 2001, University of 
California President Richard C. Atkinson proposed that the UC system stop using the SAT-I for 
college admissions and instead use a mix of achievement tests, based in large part on evidence 
that widespread SAT-I test preparation was of little educational value. Preparation for 
achievement tests (the SAT-II exams at that time) was believed to have greater pedagogical 
value. There are striking parallels in the use of the Smarter Balanced assessments versus college 
entrance tests today. The Smarter Balanced tests have been designed, from the ground up, 
with the goal of incentivizing sound classroom instruction. This is "in the DNA" for Smarter 
Balanced. To my knowledge, the same simply cannot be said of any of the college entrance 
exams envisioned in Section 1111(b)(2)(H) of ESSA, or other similar examinations that AB 751 
might make available. 

I hope that these remarks are helpful to you in your deliberations. 

With best regards, 

Edward H. Haertel, Ph.D. 

Jacks Professor of Education, Emeritus 
Graduate School of Education 
Stanford University 


cc: Lynn Lorber 

Staff Director,Senate Education Committee 


Brandon Darnell 
Principal Consultant 



